This invention relates to a low-power, single chip, parallel processor and imager system, and, more specifically, in a first embodiment, a low power, large scale MPEG2 encoder and imager system for a single-chip digital CMOS video camera is disclosed. The invention also relates to such systems having additional peripheral control processing circuitry for managing processing of the parallel processing elements and for formatting data therefrom. In connection with this aspect, a second embodiment of a lower power digital video (DV) architecture suitable for use in digital camcorders and the like is disclosed.
Processing of digital data obtained from an image sensor requires complex calculations. Processing of video data, which requires motion estimation, is particularly computationally intensive. Accordingly, various techniques have been proposed to meet these processing requirements. Thus, processors capable of performing over one billion operations per second are becoming commonplace.
A conflicting requirement for certain applications, however, is that the overall power be minimized, especially for devices such as camcorders and the like that are required to be battery powered. Thus, although the same complex calculations are required, they must be performed with a system that uses minimal amounts of power, so that the devices can operate for a reasonable period of time before requiring recharging.
Existing video processing engines are designed to optimize processing of video data stored in a secondary storage medium, e.g., random access memory, hard drive, or DVD. This results in a need for an external chipset whose primary task is to provide the necessary bandwidth for data transfer between the video engine and the secondary storage medium. The requirement of such an external data transfer eliminates the possibility for a low-power, single-chip solution.
Another existing solution that uses less power is a single integrated circuit chip for both the image sensor and digital processor. An example of such a single integrated circuit chip is the VLSI Vision Limited VV6405 NTSC Colour CMOS Image Sensor. The digital processor disclosed operates upon consecutive rows of pixel data sequentially to perform simple pixel-level computations. While this solution uses less power than other alternatives, it does not have the ability to perform operations at rates that are desired.
It is an object of the present invention, therefore, to provide an integrated image sensor and processor architecture which satisfies low power requirements.
It is a further object of the present invention to provide an integrated image sensor and processor capable of performing complex operations.
It is yet another object of the present invention to provide an integrated image sensor and processor which can output formatted image data.
It is yet another object of the present invention to provide an integrated image sensor and processor which can easily distribute processing tasks among parallel processing elements and control elements as dictated by image processing algorithms.
In view of the above recited objects, among others, the present invention implements a parallel processing architecture in which a plurality of parallel processors concurrently operate upon a different block, preferably a column, of image data. Implemented on a single monolithic integrated circuit chip, this single chip solution has characteristics that provide the throughput necessary to perform computationally complex operations, such as color correction, RGB to YUV conversion and DCT operations in either still or video applications, and motion estimation in digital video processing applications.
In a specific first embodiment according to the present invention, a parallel processor and imager system according to the present invention implements in a preferred first embodiment a single-chip digital CMOS video camera with real-time MPEG2 encoding capability. Computationally intensive operations of the video compression algorithms can be performed on-chip, at a location right beside the output of the imager, resulting in low latency and low power consumption. In all embodiments, this architecture takes advantage of parallelism in image processing algorithms, which is exploited to obtain efficient processing.
In another embodiment of the invention, a low-power, large-scale parallel digital video encoder suitable for use in a single-chip digital CMOS video camera or the like provides, in addition to basic functionality similar to that above, formatting and streaming of compressed output image data. By internalizing data transfers and compressing the exported data, the system exhibits lower power consumption than comparable multi-chip implementations which transfer large amounts of raw, uncompressed data between chips. Further, by adding peripheral processing capabilities, the processing load per image column is reduced, thereby leading to a lower clock rate and supply voltage which results in a further reduction in power consumption.